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(54) Adaptive voice-controlled dialogue system 

(57) An adaptive speech-controlled dialogue sys- 
tem (1) is described, which is arranged for adapting a 
dialogue characteristic to specific wishes of a specific 
user based on experiences with respect to this specific 
user. The system can be in a predetermined number of 
dialogue states (DSi), wherein each dialogue state (DSi) 
is associated with predetermined dialogue actions, in- 
cluding predetermined transition possibilities (TR(i;j» to 



predetermined other dialogue states (DSj). In at least 
one dialogue state, the system has certain communica- 
tion options with respect to the communication with the 
user, wherein the system adapts itself in an adaptive 
way to apparent wishes of the user concerned. 

Particularly, the system is arranged for developing, 
for at least one user (U), a user profile which defines for 
the said communication options the preferences asso- 
ciated with that said user (U). 
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Description 

[0001] The present invention relates in general to a 
speech-controlled dialogue system, with which is meant 
a computer system that can exchange data with a user 
and can receive instructions and data in dialogue form, 
especially by means of spoken text. 
[0002] Computer systems for recording data and later 
retrieval of data are commonly known. Such a system 
has in general a communication interface for two-way 
communication with a user. For communication from us- 
erto system, hereinafter indicated as "input", such com- 
munication interface comprises input means, and for 
communication from system to user, hereinafter indicat- 
ed as "output", such communication interface compris- 
es output means. Conventionally the output means are 
primarily a display and a printer, while the input means 
are, conventionally, primarily a keyboard. 
[0003] For increasing the ease of use, speech-based 
communication interfaces have recently been devel- 
oped. Herein, the input means comprise a microphone 
for receiving spoken text from a user as well as a 
speech recognition module, which converts the sound 
signals received from the user into signals understand- 
able for the system, such as for instance text or control 
instructions. Conversely, the output means can com- 
prise a speaker or the like for generating sound signals, 
as well as a speech generation module generating 
speech signals. Then, for instance, a user may, instead 
of reading a written message "close file (yes/no)?" on 
his screen, hear a spoken text "do you want to close the 
file?", and he may, instead of typing the instruction "yes 
[enter)", pronounce the instruction "yes". Since speech- 
based communication interfaces are known per se, for 
instancefrom US-A-5.051 .924 or US-A-5. 168.548, they 
will not be explained further here. 
[0004] Although a speech-controlled dialogue system 
may already be used in the case of only one single user, 
and the present invention relates also to such a situa- 
tion, the present invention more particularly relates to a 
speech-controlled dialogue system with a targe number 
of users that can input and/or request information 
through that system. An important example of an appli- 
cation situation is a hospital, where it must be possible 
for medical files to be looked into and/or to be modified 
at different places by different persons. The present in- 
vention relates particularly, but not exclusively, to such 
an application situation, and will therefore be explained 
hereinafter for this application example. However, it is 
stressed that the invention can also be applied to other 
areas, such as for instance an insurance office, an ad- 
ministration office, etc. 

[0005] Existing systems have the disadvantage that 
they follow a dialogue with a user according to a prede- 
termined and fixed protocol. The protocol therefore, is 
not easy or convenient in use for all users to the same 
extent. In any case, each user must get used to a non- 
changing system. 



[0006] The present invention aims to solve this prob- 
lem. 

[0007] More particularly, the present invention aims to 
provide a dialogue system which provides increased 
5 convenience of use in that the dialogue followed by the 
system is better tuned to the wishes of each individual 
user. 

[0008] In principle, it would of course be possible to 
consider the dialogue system as a combination of many 

10 dialogue subsystems, wherein each subsystem is allo- 
cated to a fixed user and follows a preprogrammed di- 
alogue characteristic which is adapted to this one fixed 
user. Such subsystems could be implemented in hard- 
ware or software. A disadvantage of such an approach 

J5 is. however, that a new dialogue characteristic must be 
developed for each new user. 

[0009] The present invention chooses another ap- 
proach. According to an important aspect of the present 
invention, a dialogue system is adaptive. The dialogue 
20 characteristic is not fixed but is adapted to the specific 
wishes of the users, by the system itsetf, based on ex- 
perience relating to each user. 

[0010] In practice, this has as result that each user 
may experience the system in an individual way as be- 

25 tn g a dialogue system fitting him well, and adapted to 
his specific wishes. Instead of a user having to get used 
to a fixed system, he experiences a flexible system 
which gets used to the user. Based on experiences from 
the past, the system can even anticipate to wishes of 

30 the user. 

[0011] These and other aspects, characteristics and 
advantages of the present invention will be explained 
further by the following description of an application ex- 
ample of a dialogue system according to the invention 
35 with reference to the drawing, in which: 

figure 1 shows a block diagram of a dialogue sys- 
tem: and 

figure 2 illustrates dialogue states and transitions 
40 there between. 



[0012] Figure 1 schematically shows a dialogue sys- 
tem indicated generally with the reference number 1 , ap- 
plied in the context of a hospital 2 for processing medical 

45 tiles A large number of users U , only three of which are 
shown in figure 1 , may be connected to the system 1 . A 
user can be connected directly to the system 1 by means 
of a terminal 3 arranged within the hospital 2, such as 
shown for a first user U1 . Such a user will be indicated 

so as "internal" user. However, the system 1 is also acces- 
sible for external users. In figure 1, a second user U2 
(for instance a physician) is shown, which is connected 
to the system by means of a telephone network 4, and 
a third user U3 is shown, which is coupled to the system 

55 by means of the internet 5. However, it will be clear that 
a dialogue system may offer one or more of said cou- 
pling possibilities simultaneously. 
[0013] The dialogue system t proposed by the 
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present invention can be in a predetermined number of 
dialogue states, which will be indicated hereinafter as 
DSL wherein i is an integer larger than zero. The number 
of possible dialogue states DSi is large, but finite. Each 
dialogue state DSi is characterized by one or more of 
the following dialogue actions to be performed by the 
system: 

a) Presenting information to the user. 

b) Posing a question to the user. 

c) Receiving input from the user. 

d) Taking an action in response to the input (c). 

e) Going to a next dialogue state in response to the 
input (c). 

[0014] The dialogue actions (a) and (b) concern an 
information transfer from the system to the user. In ac- 
tion (a), the information may for instance be presented 
on a display or a screen, but the information may also 
be presented auditively in the form of speech. In action 
(b), a question can be posed in written form, for instance 
by projection on a screen, but preferably a question is 
posed auditively in the form of speech. 
[0015] The dialogue action (c) concerns an informa- 
tion transfer from the user to the system. The input can 
concern data to be processed, but may also be a com- 
mand or a question, respectively; preferably, input takes 
place through spoken text. 

[001 6] The dialogue action (d) concerns an action not 
necessarily involving information transfer from and/or 
towards the user. 

[0017] A transition from a dialogue state DSi to a next 
dialogue state DSj will be indicated as transition TR(i;j). 
[0018] Figure 2 illustrates schematically a number of 
dialogue states DS1 to DS6 in the form of circles, and 
possible transitions in the form of arrows. Next to the 
arrow which connects dialogue state DS1 with dialogue 
state DS4 f the indication TR(1;4) is shown; the other 
transition indications are left out for the sake of simplic- 
ity. It appears clearly from figure 2 that the system 1 can 
reach a certain dialogue state (for instance DS4) from 
different previous dialogue states (for instance DS1 ; 
DS2). Further it appears clearly from figure 2 that, from 
a certain dialogue state (for instance DS1) : the system 
has in principle a plurality of transitions available (for in- 
stance TR(1:4) r TR(1;5) ; TR{1;6)) for reaching a next 
dialogue state (DS4, DS5 ; DS6 : respectively); the sys- 
tem will decide, based on the input received from the 
user, which transition is actually made. 
[0019] In many dialogue states, the system has cer- 
tain communication options regarding the communica- 
tion with the user. Regarding the information transfer 
from the system to the user (the dialogue actions (a) and 
(b)) : the system has for instance the following options: 

which information to present or which question to 
pose, respectively; 

which order to keep when presenting information or 



posing questions, respectively. 

[0020] Regarding the information transfer from the us- 
er to the system (the dialogue action (c)), the system 
5 has for instance the choice of how a particular request 
must be interpreted. 

[0021] Further, in many dialogue states, the system 
has certain action-options regarding an action to be tak- 
en (dialogue action (d)) or a transition to be executed 

10 (dialogue action (e)). 

[0022] According to an - important aspect of the 
present invention, the dialogue system 1 is, at least in 
at least one dialogue state, not fixed in executing the 
above-mentioned options but the system conforms itself 

is jn an adaptive way to the apparent wishes of the user 
concerned. More particularly, the system remembers 
which choices the user has made in the past in the same 
circumstances, and the system considers that the user 
now wants to make the same choice. Thus, the system 

20 develops for each user a user profile which defines for 
the above-mentioned option preferences associated 
with a certain user. During each dialogue session be- 
tween system and user, first the identity of the user is 
established by the system, the user profile associated 

25 with that user is activated, and that activated user profile 
is used for making choices. Further, if necessary, the 
user profile will be amended. 

[0023] For recording and remembering the user pro- 
file, the system comprises a memory which is not shown 

30 for the sake of simplicity. 

[0024] Hereinafter, the present invention will be fur- 
ther explained in the light of some examples, wherein it 
will be assumed that the communication between user 
and system takes place through speech, and wherein 

35 always will be assumed that no speech confusion aris- 
es. Further, the dialogue system will be indicated by the 
character S and the user will be indicated with the char- 
acter U. 

[0025] In a first dialogue state, the identity of the user 
40 is examined. For instance, that can be done as follows: 

U: "new user*'. 

S: "what is your name?" 

U: "Janssen". 

45 

[0026] Then, in a second dialogue state, the system 
can execute a verification step, wherein some personal 
data of the user are mentioned, and wherein the user is 
asked for confirmation, and finally the system can ask 
50 a password or verification code. For reasons of security 
and for protecting the privacy of the user, these data are 
preferably not exchanged by means of speech but in a 
way which is not easily overheard by those accidentally 
present, for instance through screen and keyboard. 
55 [0027] When the system has established the identity 
of the user, the system enters a third dialogue state, 
wherein the user is asked what he now wants to do, for 
instance as follows: 
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S: "what do you want to do?". 

[0028] There is. of course, a large amount of possibil- 
ities for the user but for the sake of simplicity it will be 
assumed that the user can chose from the following ac- 
tions: 

1) making an examination report; 

2) examining lab results; 

3) writing a report letter; 

4) writing a referral letter; 

5) requesting an examination. 

[0029] Suppose that the user is a family doctor who 
wishes to make a referral letter, and who answers as 
follows: 

U: "letter". 

[0030] Here, the system finds insufficient information 
in the input for making a choice: the possibilities (3) and 

(4) are the most probable, but the possibilities (1) and 

(5) cannot be excluded. Then, the system is forced to 
obtain further information. 

[0031] Conventionally, a computer program returns to 
a question step if the answer to the question does not 
meet the criteria set by the program. In this case, this 
would mean that the system would repeat its question, 
and that the above-described cycle of question and an- 
swer can stay repeating itself. 

[0032] In order to make clear to the user what is ex- 
pected, the dialogue system might follow the options 
one by one in a fixed order, and ask if that is the right 
choice, as follows: 

S: "do you want to make an examination report, yes 

or no?" 
U: "no" 

S: "do you want to examine lab results, yes or no?" 
U: "no" 

S: "do you want to write a report letter, yes or no?" 
U: "no" 

S: "do you want to write a referral letter, yes or no?" 
U: "y es " 

[0033] Of course, this cycle of questions does lead to 
the result that the wish of the user is made clear such 
that the system can go on to a next dialogue state, but 
it is inconvenient to the user that onty the fourth question 
is a hit. In practice, of course, the number of possibilities 
is larger, and so it can take longer before the system 
reaches the possibility wished by the user. 
[0034] According to an important aspect of the 
present invention the system is adapted for remember- 
ing the answer of the user, and for incorporating the an- 
swer in the user profile belonging to this user U. When 
this same user U reports a next time, the system will 
pose its question, based on earlier experiences record- 



ed in the user profile. After all, apparently this user U 
meant "referral letter" with the command "letter" and the 
chances are that is what he means now again. A next 
time, the system will pose this question first. 

5 [0035] The system may be adapted for adapting each 
choice to the previous meaning of the command con- 
cerned, but preferably the system is adapted to adapt 
its choice to the largest number of previous meanings 
of the command concerned. If, after six "referral letters", 

w the user writes a report letter ones, a next time the sys- 
tem will still assume that the command "letter" means 
"referral letter". 

[0036] After a learning phase, in which the system 
"gets used" to the user, the system will "recognize" this 

15 command "letter" better and better in a way adapted to 
the user concerned. With a different user, the recogni- 
tion can progress differently. Suppose that a second us- 
er exists, a surgeon who, with "letter, always means 
"report letter"; in time, the system will associate the com- 

20 mand "letter" with the meaning "report letter" with this 
second user, and will incorporate that in the user profile 
belonging to this second user. 

[0037] In this way, all users are offered the important 
convenience of use that the dialogue system "under- 

25 stands" what the user means by certain instructions. 
[0038] In the end it may even be so that the dialogue 
system recognizes with the first-mentioned user (gen- 
eral practitioner) that he will almost always request a 
referral letter, and only seldomly something else. Then, 

30 the dialogue system can adapt its first question in this 
dialogue state to this, for instance by. instead of "what 
do you want to do?*', posing as first question: 

S: "do you want to make a referral letter again?" 

35 

[0039] Another example concerns a specialist who 
wants to refer a patient to a radiologist for making an X- 
ray photo. Then, the system enters a dialogue state in 
which further information is asked about the X-ray photo 
40 to be made; in the referral letter must, inter alia, be in- 
cluded from which part of the body the photo must be 
made. The system can ask the specialist for information, 
as follows: 

45 S: "which part of the body?" 
U: "lungs" 

[0040] It will however be clear that a lung specialist is 
generally more interested in lung photos than in photos 

50 from other body parts such as knees. Because the lung 
specialist primarily chooses for lung photos, the system 
according to the present invention incorporates this 
preference in the user profile of this user/lung specialist, 
and it will adapt the dialogue thereto. Then, for instance, 

55 the dialogue can be: 

S: "lung photo?" 
U: "yes" 
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[0041] An important advantage according to the 
present invention is the increased ease of use in that 
exchange of information takes place by means of 
speech , at least in as far as it concerns the direct transfer 
from and towards a user. In the system itself and/or in 
a communication network, the transfer can take place 
by means of other more suitable coding. For instance, 
in the case of communication through internet the 
speech signals can be converted into HTML codes. 
[0042) Generating speech by a system for communi- 
cation with the user can be performed in several ways. 
By way of example it is possible that a sequence of char- 
acters (letters, numerals), for instance ASCII-charac- 
ters, generated by the system, are converted into 
speech signals by a speech converter at the place ol the 
user. However it is also possible that use is made of 
preprogrammed speech sequences, which can be acti- 
vated by the system as desired by sending a regarding 
activation code ("canned speech"). An advantage asso- 
ciated with this latter method is that the system can con- 
fine itself to generating a relatively short code (for in- 
stance: code=1) : while at the place of the user a rela- 
tively long sentence, containing much information, is 
generated (for instance: "what do you want to do?"). 
[0043] A further important advantage of the system 
according to the present invention is the flexible inter- 
action with the user. In the foregoing it has already been 
explained that the system bases itself, when making 
choices, on the user profile associated with the user. 
Further the system is very flexible in accepting input by 
the user More particularly, the user is not obligated to 
formulate his commands and data input according to a 
predetermined format. In the foregoing, it has been de- 
scribed by way of example that a user reports himself 
with the message "new user". However the user is not 
bound to report himself in this way. The system is ar- 
ranged for within certain limits, understanding the se- 
mantics of the user's command, for instance by recog- 
nition of certain key words (for instance: "I am a new 
user"), or recognition of equivalence (for instance: "I 
want to begin"). Such key words and/or equivalence can 
be preprogrammed. However also in this respect it is 
possible that the system adapts itself to the user by in- 
corporating earlier non-understood input in a table of 
equivalence in the user profile alter elucidation. A user 
who reports himself with for instance the text "hello", will 
maybe not be understood directly. By means of a cycle 
of question/answer the system can then learn the inten- 
tion of the user By regarding the association between 
this intention and the text "hello", a next time this user 
reports himself with the text "hello" the system will au- 
tomatically be able to interpret this as being a possible 
equivalent for "new user". 

[0044] Since it will be clear to persons skilled in the 
art how the system according to the present invention 
can be implemented by suitable software and/or hard- 
ware, this aspect will not be explained here further 
[0045] It will be clear to a person skilled in the art that 



8 

the scope of the present invention is not limited to the 
examples discussed in the foregoing, but that several 
amendments and modifications thereof are possible 
without deviating from the scope of the invention as de- 
5 fined in the attached claims. 



Claims 

io 1. Adaptive speech-controlled dialogue system (1), 
comprising a speech-based communication inter- 
face through which a user (Ul; U2; U3) can receive 
and input data and can input instructions, wherein 
the system is arranged for adapting a dialogue char- 
's acteristicto specific wishes of a specific user based 
on experience in relation to this specific user 

2. Dialogue system according to claim 1 , wherein the 
communication interface uses a network such as a 

20 telephone network (4) or the internet (5). 

3. Dialogue system according to claim 1 or 2, wherein 
the system can be in a predetermined number of 
dialogue states (DSi), wherein each dialogue state 

25 (DSi) is associated with predetermined dialogue ac- 
tions, including predetermined transition possibili- 
ties (TR(i;j)) to predetermined other dialogue states 
(DSj). 

30 4. Dialogue system according to claim 3, wherein, in 
at least one dialogue state, the system has certain 
communication options with respect to the commu- 
nication with the user. 

35 5. Dialogue system according to claim 4, wherein, at 
least in respect of at least one of said dialogue 
states where the system has communication op- 
tions, the system adapts itself in an adaptive way to 
apparent wishes of the user concerned. 

40 

6. Dialogue system according to claim 5, wherein the 
system is arranged for, for at least one user (U), de- 
veloping a user profile which defines for the said 
communication options the preferences associated 

45 with that said user (U). 

7. Dialogue system according to claim 6, wherein the 
system is arranged for at least in respect of the said 
one dialogue state, incorporating an answer or 

so choice given by the user into the user profile belong- 
ing to this user (U). 

8. Dialogue system according to claim 6 or 7, wherein 
the system is arranged for posing questions in a cer- 

55 tain situation based on the earlier experiences with 
respect to the same situation embedded in the user 
profile. 
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9. Dialogue system according to claim 6 or 7, wherein 
the system is arranged for interpreting an instruc- 
tion of the user (U) in a certain situation based on 
the earlier experiences with respect to the same sit- 
uation incorporated in the user profile. 5 

10. Dialogue system according to claim 8 or 9, wherein 
the system is arrangedfor incorporating into the us- 
er profile exclusively the last previous experience 
with respect to a certain situation. 10 

1 1 . Dialogue system according to claim 8 or 9 : wherein 
the system is arranged for embedding into the user 
profile a predetermined number of previous experi- 
ences with respect to a certain situation, and to « 
base its choice on the largest number of experienc- 
es incorporated in the user profile. 

20 
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